Recent innovations with respect to Graphics Processing Units (GPUs), and graphics cards hosting GPUs, have enabled increasingly fast image compositing operations. Turning first to two-dimensional compositing operations, these can now typically be performed using shading languages designed to run on a GPU. For example, given a set of images, a compositing order, and a compositing operation for each image, the GPU enables the copying of a background image from a frame buffer to a video memory (e.g., one or more texture processing units) of the GPU. The GPU may then use the background image and associated alpha channel information, along with a foreground texture and associated alpha channel information, to produce another background image in the frame buffer. This process is repeated by the GPU for each foreground image. Complications can arise because of the limited resources of the GPU, and the time required to transfer images to the video memory. Many techniques are employed to minimize the transfer time and storage requirements.
Turning now to three-dimensional compositing, in contrast to two-dimensional compositing, three-dimensional compositing typically presents no fixed sorting order. Instead, the order within each compositing operation needs to be determined from the geometrical position of images (or image elements) within a three-dimensional space. For example, if a ray from a view point passes through two image elements, the first image element “hit” by the ray is the foreground, and the second image element is the background. Complications arise when images intersect, and no clear sorting order exists. In view of the difficulties that are presented with respect to three-dimensional compositing on a GPU, compositing three-dimensional images has typically been performed on a Central Processing Unit (CPU) of a computer system. For example, the Painter's algorithm has been utilized to handle non-intersecting image layers. For intersecting layers, the A-buffer technique has been utilized on the CPU to sort image fragments. A third technique that has been deployed on the CPU is to split images into non-intersecting pieces, and use the Painter's algorithm on these pieces. On broadcast-sized images, these techniques, when implemented on the CPU, are not fast enough to be considered real time.
To accelerate compositing operations, developers have attempted to use the GPU (e.g., through the OpenGL and Direct3D interfaces) to perform certain compositing operations. However, blending equations provided by the interfaces are insufficient to handle a wide variety of compositing modes.
Turning now to GPU processing, certain techniques (e.g., the Painter's algorithm) may run well on the GPU but are unable to process intersecting images. The above-mentioned A-buffer technique, where each pixel in the frame buffer contains a depth-sorted list of fragments, is not well suited for their GPU. Further, typical techniques to render three-dimensional transparent, non-intersecting image layers on the GPU typically require multiple passes through the data, resulting in performance penalties.
In short, previous GPU-assisted techniques for compositing three-dimensional images handled non-intersecting layers satisfactorily, and also satisfactorily computed lights, shadows and other effects. However such techniques did not handle intersections correctly, and only allowed the use of binary trackmattes. Such techniques also required multiple passes of the data. Such additional passes through a three-dimensional scene, to process multiple effects (e.g. lighting, trackmattes, motion blur, etc.), resulted in reduced performance.